This application relates to the field of computer networks, and specifically to software and hardware for monitoring and controlling network traffic. Computer networks often include hundreds or thousands of network hosts. A network host is a computer or other hardware device that runs software applications and originates and/or receives network flows.
Network administrators are often responsible for maintaining these network hosts in proper running order and to ensure the network operates securely and reliably. To that end, network administrators often set rules or network policies about the types of software applications and network traffic allowed on a network.
Network applications are software applications on a network host that are responsible for originating and/or receiving network traffic flows, referred to as network flows. Some network applications are well-behaved and conform with a network's rules and policies. Other network applications are poorly-behaved, installing without the user's or network administrator's permission, hiding themselves and their operation, and violating a network's rules and policies. Examples of poorly-behaved network applications include computer viruses, worms, and spyware and malware applications. Additionally, some more legitimate applications, such as instant messaging applications, file-sharing or other types of peer-to-peer network applications, voice-over IP (VOIP) communication applications, and multimedia applications are responsible for network flows that can circumvent network policies and jeopardize network security and reliability.
Often, poorly-behaved network applications will attempt to conceal their network flows to avoid detection and disregard network policies. Common evasion techniques include using non-standard network protocols, dynamic port and channel selection, which limits the effectiveness of monitoring and blocking network ports to control network traffic; HTTP/HTTPS tunneling, which hides network flows in normally-permitted web traffic; Peer-to-Peer onion routing, which selects destination addresses for peer-to-peer routing at random to circumvent destination address blocking; and encryption of network packet data, which prevents network monitors from examining the contents of network packets to identify the type of network flow.
For example, some common peer-to-peer VOIP applications circumvent network policies in a number of ways. The peer-to-peer VOIP application can dynamically selected different ports and channels for communication. If UDP is blocked, the application can fall back on TCP/IP. Additionally, the peer-to-peer VOIP application can tunnel its data over open ports 80 or 443, which are normally intended for HTTP or SSL traffic. A peer-to-peer VOIP application can dynamically select supernodes in its peer-to-peer network to circumvent destination address detection and blocking. Additionally, data can be encrypted to prevent detection using packet inspection.
Prior network monitoring applications could monitor the content, size, and source and destination addresses of network flows as they pass through a gateway or other point in the network. However, due to these evasion techniques, prior network monitoring applications often have too little information to reliably identify unauthorized network flows and detect poorly-behaved network applications.
It is therefore desirable for a system and method to provide improved capabilities in categorizing and controlling network flows and network applications. It is further desirable for the system and method to detect and adapt to new or revised network applications and network flows with little or no user intervention.